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■ Abstract 

Oh: 

■ We consider moments of the return times (or first hitting times) in a discrete time 
discrete space Markov chain. It is classical that the finiteness of the first moment 
of a return time of one state implies the finiteness of the first moment of the 
first return time of any other state. We extend this statement to moments with 

p/| . respect to a function /, where / satisfies a certain, best possible condition. This 

PLh ! generalizes results of K. L. Chung (1954) who considered the functions f(n) = 

ri p and wondered "[...] what property of the power n p lies behind this theorem 
[...]" (see Chung (1967), p. 70). We exhibit that exactly the functions that do not 
increase exponentially - neither globally nor locally - fulfill the above statement. 
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A classical result, see e.g. |Kol36j . states that for any recurrent, irreducible Markov chain 
on a countable state space the following holds: if for any state % the first moment of the 
recurrence time is finite then this also applies to any other state. A first generalization of 



this result appeared in |HR53j . If we denote by ly the first time that the Markov chain 
visits state j if it is started in i, then the result can be stated as follows: E/(Tjj) < oo 
for some state % implies that Kf(Tjj) < oo for any other state j, where f(x) = x n 
for some integer n. The authors state the result as a lemma and refer to the proof to 
an (unpublished) note by K. L. Chung and R. N. Snow. A more general result can be 
found in [Chu54l Theorem 1], where / is allowed to be of the form x p , for any real p > 0. 
After stating the theorem, the author also comments that the concept of generalized 
moments defined in terms of a general function / was suggested to him by J. L. Doob, 
but Chung only mentions that his results can also be shown for functions / satisfying 
f(x + y) < f(x) + Af(y), for some constant A. 



*Institut fur Mathematik, MA 7-5, Fakultat II, Technische Universitat Berlin, Strafie des 17. Juni 
136, 10623 Berlin, Germany; {aurzada, hdoering, ortgiese, ms}@math.tu-berlin.de 



1 



Further related research considers recursive formulas for second moments in terms of 
first moments |Chu54| Sect. 2], factorial moments |Lam60j . and also, more recently, 
explicit formulas for higher polynomial moments |Sze08j . 

These considerations naturally lead to the question, for which functions it is true that 
a finite generalized moment of the return time for one state % implies that the moment 
is also finite for the return time for any other state of the Markov chain. In this note, 
we characterize this class of functions. In the following, we only consider irreducible, 
recurrent discrete time Markov with a countable state space. 

To formulate our results, we introduce the following notation. The candidate functions 
/ are taken from the set 

T := {/ : N — > (0, oo) is non- decreasing and lim f(n) = oo}. 

Then our objective is to classify the collection Q of all / G T such that for each irre- 
ducible recurrent discrete time Markov chain with a finite or countably infinite state 
space E, the following holds: if E/(T&) < oo for some i G E then Ef(Tjj) < oo for all 
j EE. 

Following Chung |Chu54j we additionally introduce the class H, by stating that / G H, 
if for any Markov chain the following holds: if there exist two states i and j such that 
Ef(Tij) < oo and E/(Tjj) < oo then E/(T^) < oo for any pair k,£. 

As the states i and j do not have to be distinct, it follows that H is contained in Q . 
The classical result due to Kolmogorov implies that the identity function belongs to % 
and |Chu54j shows that any fix) = x p , for p > 0, belongs to %. 

Our main result states that the two classes Q and % are in fact the same; and we also 
give a characterization for a function to be in this class. 

Theorem 1.1. Let f G T . Then the following statements are equivalent: 

(a) f G H; 

(b) f g G; 

(c) the following two conditions are satisfied: 

(i) there exists K > such that for any x,y > 0, f(x + y) < Kf(x)f(y), 

(ii) limsup n ^ 00 ilog/(n) = 0. 

Condition (c) has an easy interpretation: It ensures that the function / does not grow 
exponentially fast - neither globally nor locally. In fact, one can construct functions 
outside Q that globally increase as slowly as one wishes, but locally have parts of expo- 
nential increase (cf. Example 13. 3j) . 

This note is structured as follows. In Section [21 we prove the implication (c) =>• (a) of 
our main theorem. In Section [31 we prove the implication (b) =>■ (c) by showing that any 
function that violates either condition (z) or (ii) in (c) is not in the class Q. Together 
with the earlier observation that H C Q, so that (a) =^ (b), this shows the equivalence 
of the three statements. 
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2 Proof of (c) (a) 



In this section, we prove that (c) =>• (a) in our main theorem. We start with the following 
lemma, which collects some preliminary facts. 

For this purpose, it is convenient to introduce the following notation: for states i and 
j of a Markov chain, we denote by the return time from i to i conditioned on not 
crossing j (if there is such a path with positive probability). Further, denotes the 
first hitting time of state j when started from i conditioned on not returning to % before 
hitting j. 



state i 




Tu conditioned on crossing j 



Figure 1: Visualization of the relevant return/hitting times 

Lemma 2.1. Let f e T . 

(i) If for a state i of a Markov chain we have Ef(Tu) < oo then, for any state j, 
Ef(U i:j ) < oo and E/(V-j) < oo. 

(ii) If for a state i of a Markov chain we have Ef(Tu) < oo then, for any state j, 
Ef(T ri ) < oo. 

Proof. To see (i) note that if the probability p of going from from i to i without crossing 
j is positive, we have 

oo > Ef(T u ) > E[/(7yi {do not cross j} ] = E[f(T tt ) | do not cross j]p = pEf^). 

This shows Kf(Uij) < oo. 

Similarly, since q := 1 — p is the probability of first hitting to j before returning to i, 
which is positive since the Markov chain is recurrent and irreducible, we have 

oo > Ef(T u ) > E[/(T«)l {d o cross,}] = E[/(7ii) | do cross j]q> qEf(V tJ ). 

Here we used that Tu conditioned on crossing j is stochastically larger than Vij. This 
shows Ef(Vij) < oo. 

To see (ii) note that 

oo > E/(T«) > E[/(T«)l { do cross,}] = E[/(T«) I do cross j\q> gE/(T J4 ), 

where q > is as above. Here we used that Tu conditioned on crossing j is stochastically 
larger than 7}j. □ 
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We stress that, in the second part of Lemma [2~T| one cannot prove in the same way that 
E/(7y) < oo, since a typical path from % to % does not necessarily contain a path from i 
to j. However, as the next lemma shows this can be shown if we assume that condition 
(c) holds. This lemma is also the main part of the argument for the proof of (c) =>■ (a) 
in our main theorem. 

Lemma 2.2. Let f G T and assume that (c) holds. 

(i) If for two states i and j of a Markov chain we haveEf(Tij) < oo andEf(Tji) < oo 
then Kf(Ta) < oo. 

(ii) If for a state i of a Markov chain we have Ef(Tu) < oo then, for any state j, 
Ef^) < oo. 

Proof. First we show (i). Clearly Ta is stochastically dominated by ly + Tji, where T^- 
and Tji are independent. Using (c) and the monotonicity of /, we get 

E/(T«) < Efpy+Tji) < E[Kf(Tij)f(Tji)] = ^E/(X tJ )E/(T rt ) < oo. 



Now we turn our attention to (ii). For the purpose of this proof, define /(0) := 1/K, 
where K is as in (i) of (c). Note that /(l) < f{2) < Kf(l) 2 , so that f(0) = K~ l < /(l). 
This shows that in fact f(x + y) < Kf(x)f(y) for all x, y > 0, since f(x + 0) = f(x) = 
Kf(0)f(x). 

The crucial observation (also cf. (I3.3P below) is that 



M 



r=l 



where the random variables U^ r ' are i.i.d. copies of the random variable Uij as defined be- 
fore Lemma 12.11 and V is a copy of the random variable Vij as defined before Lemma 12.11 
and all variables are independent. Further, M (independent of the U's and V) is a geo- 
metric random variable with mean — where ir > is the probability of first hitting 
j before i when started from i. 

It may be that 7T = 1 - which is the case if and only if there is no path from i to i 
without crossing j - in which case = V and we are already done. Excluding this 
case, we derive from (c) and (12.11) that 

M oo m 

Efftj) < KEf(j2U {r) )Ef(V) = K^l - n) m 7rEf(j2u {r) )Ef(V). 

r=l m=0 r=l 

Since by Lemma l2.1( i) E/(V) < oo, it is clear that the last expression is finite provided 
that 

lim sup — log Ef ( V U ir) ) = , (2.2) 

r=l 

where we know that Ef^U^) < oo from Lemma [2.1( i). 
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To show (12. 2p fix a large constant A > and estimate using (c) as follows: 

m 

Ef{U {1) + ... + U (m) ) < E/ ( JZ(t/ (r) V A)) 

r=l 
m 

<Ef(j2(U {r) l {U (r )>A} + A) 

r=l 

m 

<Kf(Am)Ef(j2u ir) * { uV >A} 

r=l 

m 

< Kf(Am)E[K m - 1 Hf(u^l {uM>A} 

r=l 

= f(Am)E[Kf(u^t {uW>A} 
Hence, 

-logE/([/« + ... + C/M) < Ilog/(Am)+log(KE/(C/«l {c/ ( 1)>A} )). 
Letting m — >• oo and using part (ii) of (c) we get 

limsup - logEf(U^ + ... + U^) < log(KEf(U^l {um>A} )) . 

m—^oo Tfl 

Letting now A — > oo and using dominated convergence since Ef(U^) < oo, we have 

limsup — log Ef{U {1) + ... + U {m) ) < \og(Kf(0)) = 0, 

which proves (12.21) . □ 
Now we can prove that (c) =>- (a) in our main Theorem 11.11 

Proof of (c) =>- (a) in Theorem 11.11 Consider a Markov chain with state space E, 
let / satisfy condition (c) of Theorem ll.il and assume that i,j e E satisfy E/(Tjj) < oo 
and Ef(Tji) < oo. Let k,£ G E. We want to show that E/(T^) < oo. 

By Lemma l2.2( i). we have E/(2#) < oo and Ef(Tjj) < oo. Thus, by Lemma l2.2( ii) 
EJ(T^) < oo. And by Lemma O^ii) E/(T W ) < oo. 

Therefore, using once again part (i) of (c), we have 

Ef(T ke ) < Ef{T kl + Tij + T j( ) < K 2 Ef(T ki )Ef(Tij)Ef(Tj e ) < oo. 

□ 



3 Proof of (b) (c) 

In this section, we will show the implication (b) =>■ (c) in Theorem 11.11 by showing 
that the conditions of subexponential growth rate and submultiplicativity are in fact 



5 



(L,n,2) (L,n,l) \ / (R,n,l) (R,n,2) 




Figure 2: Transition graph of the Markov chain 

necessary. In Lemma 13.11 we give abstract conditions on /, which imply that / ^ 
Q, which we exploit in Lemma 13.21 to show that if / violates the submultiplicativity 
condition (i) of (c) in the main theorem, we have that / ^ Q. Finally, in Lemma l3~3l we 
show that any function growing exponentially fast, i.e. does not satisfy condition (ii), 
does not belong to Q. 

Lemma 3.1. If for a given f G T one can construct two independent random variables 
Ui and U2 taking values in N with infinite support such that E/([Zj) < 00 for i = 1,2, 
but E/(Z7i + U 2 ) = 00, then f (£ Q. 

Proof. Given the two random variables, we will construct a Markov chain with two 
special states and 1 with the property that E/(Tn) < 00, whereas Ef(T 00 ) is infinite, 
which shows that / ^ Q. The construction of the Markov chain is in some regards 
similar to |YK39] . where the authors construct a Markov chain with T 00 having any 
particular distribution. 

Denote by {xi,X2, ■ ■ ■ }, {2/1, 2/2, • • • } the support of U\ and U2, respectively. Formally, 
we can write the state space E of our Markov chain as 

{0}U{l}U{(L,n, m) : n G N,m = 1, . . . ,x n -l}u{(R,n,m) : n G N, m = 1, . . . ,y n -l}. 

The state is connected only to 1, and if in state 0, the chain always moves to 1 next, 
i.e. poi = 1. If in state 1, the chain has three possibilities. The first one is that the 
chain moves to with probability p, for some parameter p G (0, 1). Then, conditionally 
on not going to 0, with equal probability it either moves "left" (i.e. to a state (L,n, 1)) 
or "right" (i.e. to a state (R, n, 1)). 

Conditionally on the next move going to the "left", we want T n to have distribution 
Ui, therefore we set 

1 — p 

Pl,(L,n,l) = — 2~ = X n ) , 
P(L,n,m-l),(L,n,m) = 1, Tl G N, 1 < 771 < X n , 
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where we identify (l,n,x n ) with 1. Similarly, conditionally on the next move going 
"right", we would like T n to have the distribution U 2 , and set 

1 — p 

Pi,(R,n,i) = -^—^(U 2 = y n ) , 

P(R,n,m-l),(R,n,m) =1, H6N,l<m<l/„, 

where we again identify (R,n,y n ) with 1. 

Now, we can calculate the generalized moments of T 00 and T u . Firstly, we find for T n 
by conditioning on the three different possibilities 

E/(Tu) = /(2)P{1 ->■ 0} + E/(C/i)P{l ->■ "left"} + Ef(U 2 )¥{l -> "right"} , 

which is finite by our assumptions on and f/ 2 - 
However, for T 00 , we obtain 

M 

E/(Too)=E/ + 2), (3.3) 

i=l 

where M is a geometric random variable with parameter p and C/}q are independent 
random variables that have the same distribution as Tn conditioned on not going to 
in the first step. (Note the relation of (13. 3p and (12.11) .) In particular we obtain a lower 
bound by considering the following strategy: first the Markov chain jumps from to 
1, then it takes a tour to the "left" and after that it takes a tour of "right", before it 
finally returns to 0. Thus, we obtain using that / is non-decreasing 

E/(Too) > E[/(2 + U$ + U$) I first left, then right ] P{ first left, then right } 
>Ef(U l + U 2 )p{ 1 -^) 2 , 

where the latter is infinite by our assumptions on U\ and U 2 \ and thus as claimed E/(T o) 
is infinite. □ 

The next lemma uses the construction in Lemma 13.11 to show that any function not 
satisfying the submultiplicativity condition (i) in (c) is not in Q. 

Lemma 3.2. Suppose that f £ J 7 is such that for any C > 0, there exist xq and yc 
such that 

f(xc + yc) > Cf(x c )f(yc), 

then f ^ Q . 

Proof. By Lemma 13.11 it suffices to construct two random variables U\ and U 2 such 
that Ef(Ui) < 00 for % = 1, 2 and Ef(Ut + U 2 ) = 00. 

By our assumption on /, we can find increasing sequences (xk)k>i and (yk)k>i such that 

f(x k + yk) > k 6 f(x k )f(y k ) . 
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Indeed, to see the existence of such sequences assume that for all x > Xk + 1 and 
V > Uk + 1 the following inequality holds 

f(x + y)<(k + lff(x)f(y). 

A short calculation implies that 

f(x + y)<(k + l) 6 ( /(Xfc J ( ^ + 1) ) 2 f(x)f(y) for all x, y > , 
in contradiction to the assumption of the lemma. 

Then, for defmiteness, let Ui be a random variable taking value Xk with probability 
Pfc := Ci/(xfc) _1 A; _2 (with a suitable normalizing constant C\) and similarly, f/2 takes 
value yu with probability qu ■— C2f{yk)~ 1 k~ 2 (with a suitable normalizing constant C2). 
In particular, we find that ~Ef(Ui) is finite for i — 1,2. However, 

e/(£/i + c/ 2 ) > X] + > c ic 2 ^fMf^fM^k- 2 !^)- 1 ^ 2 , 

fc>l fc>l 
which is infinite. □ 

The following example exhibits a "typical" function that can be chosen to satisfy (ii) of 
(c), while not obeying (i) of (c). 

Example 3.3. We now construct a function /, where we can choose the parameters in 
such a way that the condition in Lemma [3.21 is satisfied and limsup - log/(n) = 0. We 
first describe the function g = log/. Take two sequences (s»)i>i and (w*)i>i such that 
Ui < s i+ i — Si and u n — > 00 as n — > 00. Then, by setting g(Q) = 0, s = u = and for 
% > 0, define g to be constant on the interval (sj + itj, Sj + i) and for i > 1, assume that on 
the interval (sj, Sj + itj) the function (7 grows linearly with slope 1. Then, by adjusting 
the parameters Ui, Si (in such a way that lim^oo g{n)/n = 0) one can make sure that 
the condition in Lemma [3721 is fulfilled for f = e 9 , while at the same time by making the 
differences Sj — Sj_! large enough, one can let / grow as slowly as desired. 

The following lemma shows that the condition on the subexponential growth rate of / 
- (ii) of (c) - is really necessary. 

Lemma 3.4. Suppose that f G J 7 satisfies 

lim sup — log f(ri) > , 

n— >oo Tl 

then f $lQ. 

Proof. By our assumption on / we can find an increasing sequence Xi and 5 > 0, 
such that f(xi) > e Sxi . Then, consider the Markov chain with two states and 1 and 
transition probabilities poi = V an d P10 = 1 for some parameter p 6 (0, 1). Now, E/(T o) 
is finite, while for any % > 1, 

E/(Tn) > e Sxi F{T n > x % } = e Sx *(l - p) x >~ 2 , 

which tends to infinity as i — > 00 provided that p is sufficiently small. □ 
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